Smart building management suite using robot command and control

ABSTRACT

A computer-implemented method includes displaying a bot client application including an active channel indicator, receiving bot input, analyzing the bot input, generating an output, and displaying the output. A smart building system includes a processor and a memory storing instructions that, when executed by the one or more processors, cause the system to display a bot client application including an active channel indicator, receive bot input, analyze the bot input, generate an output, and display the output. A non-transitory computer readable medium containing program instructions that when executed, cause a computer to display a bot client application including an active channel indicator, receive bot input, analyze the bot input, generate an output, and display the output.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 16/923,859,entitled AVAILABLE VEHICLE PARKING SPACE DISPATCH, filed on Jul. 8,2020. Application Ser. No. 16/923,859 is a continuation of applicationSer. No. 16/823,849, entitled SMART BUILDING MANAGEMENT SUITE USINGROBOT COMMAND AND CONTROL, filed on Mar. 19, 2020, now U.S. Pat. No.10,789,846, granted on Sep. 29, 2020. Each of the foregoing areincorporated herein by reference in their respective entireties.

TECHNICAL FIELD

The present disclosure is generally directed to methods and systems forperforming building automation, and more particularly, to techniquessmart building management using robot command and control.

BACKGROUND

Building management is complicated, as buildings conventionally lackautomation. As a result, building owners see significant costsassociated with a variety of problems relating to the physicalinfrastructure of the building. For example, visitors who are unfamiliarwith the building may have difficulty locating building tenants.Visitors and tenants have difficulty locating parking spots (e.g., forelectric vehicles) and finding office locations (e.g., conferencerooms). Tenants may leave electric lighting on at times when rooms areunoccupied, relating in higher costs and wasting power. Further,conventional building security is not sufficient.

Technology-focused companies increasingly deploy communicationsplatforms (e.g., Slack, Microsoft Teams, Cisco Webex, etc.) in theirorganizations, and users are increasingly integrating such platformsinto their daily workflows. However, conventional communicationsplatforms lack building management functionality.

BRIEF SUMMARY

In one aspect, a computer-implemented method for smart buildingmanagement using robot command and control includes (i) displaying, viaa graphical user interface, a building automation bot client applicationincluding an input field and an output field; (ii) receiving, via theinput field, bot input from a user; (iii) analyzing the bot input fromthe user using a trained machine learning model to identify a botinstruction; (iv) analyzing the bot instruction to identify a userintent; and (v) causing, based on the user intent, a building automationtask to occur.

In another aspect, a smart building management system includes one ormore processors; and a memory storing instructions that, when executedby the one or more processors, cause the system to: (i) display, via agraphical user interface, a building automation bot client applicationincluding an input field and an output field; (ii) receive, via theinput field, bot input from a user; (iii) analyze the bot input from theuser using a trained machine learning model to identify a botinstruction; (iv) analyze the bot instruction to identify a user intent;and (v) cause, based on the user intent, a building automation task tooccur.

In yet another aspect, a non-transitory computer readable mediumincludes program instructions that when executed, cause a computer to:(i) display, via a graphical user interface, a building automation botclient application including an input field and an output field; (ii)receive, via the input field, bot input from a user; (iii) analyze thebot input from the user using a trained machine learning model toidentify a bot instruction; (iv) analyze the bot instruction to identifya user intent; and (v) cause, based on the user intent, a buildingautomation task to occur.

BRIEF DESCRIPTION OF THE FIGURES

The figures described below depict various aspects of the system andmethods disclosed therein. It should be understood that each figuredepicts one embodiment of a particular aspect of the disclosed systemand methods, and that each of the figures is intended to accord with apossible embodiment thereof. Further, wherever possible, the followingdescription refers to the reference numerals included in the followingfigures, in which features depicted in multiple figures are designatedwith consistent reference numerals.

FIG. 1 depicts an exemplary computing environment in which methods andsystems for performing building automation may be implemented, accordingto one embodiment.

FIG. 2A depicts an exemplary building automation graphical userinterface web application, according to one embodiment.

FIG. 2B depicts an exemplary building automation graphical userinterface bot viewer, according to one embodiment.

FIG. 3A depicts exemplary building automation e-receptionist data,according to one embodiment.

FIG. 3B depicts an exemplary building automation e-receptionist logingraphical user interface, according to one embodiment.

FIG. 3C depicts an exemplary building automation e-receptionistregistration graphical user interface, according to one embodiment.

FIG. 3D depicts an exemplary building automation e-receptionist photoregistration graphical user interface, according to one embodiment.

FIG. 3E depicts an exemplary building automation e-receptionistregistration submission graphical user interface, according to oneembodiment.

FIG. 3F depicts an exemplary building automation bot notificationgraphical user interface, according to one embodiment.

FIG. 3G depicts an exemplary building automation email notificationgraphical user interface, according to one embodiment.

FIG. 3H depicts exemplary building automation registration log data,according to one embodiment.

FIG. 4A depicts exemplary building automation motion detectionnotification, according to one embodiment.

FIG. 4B depicts exemplary building automation badge registrationgraphical user interface, according to one embodiment.

FIG. 4C depicts an exemplary computing system for implementingmulti-factor badging, according to one embodiment.

FIG. 4D depicts an exemplary multi-factor prompt graphical userinterface, according to one embodiment.

FIG. 4E depicts an exemplary multi-factor prompt graphical userinterface, according to one embodiment.

FIG. 4F depicts an exemplary multi-factor prompt graphical userinterface, according to one embodiment.

FIG. 4G depicts an exemplary multi-factor prompt graphical userinterface, according to one embodiment.

FIG. 5A depicts an exemplary bot command graphical user interface,according to one embodiment.

FIG. 5B depicts an exemplary bot command graphical user interface formap querying, according to one embodiment.

FIG. 5C depicts an exemplary map graphical user interface for displayingmap query results, according to one embodiment.

FIG. 5D depicts an exemplary bot command graphical user interface formap querying, according to one embodiment.

FIG. 5E depicts an exemplary mixed-use parking lot, according to oneembodiment.

FIG. 5F depicts an exemplary bot command graphical user interface forparking querying, according to one embodiment.

FIG. 5G depicts an exemplary bot command graphical user interface forparking querying, according to one embodiment.

FIG. 5H depicts an exemplary parking space request queue, according toan embodiment.

FIG. 6A depicts exemplary data logs for tracking room presence,according to an embodiment.

FIG. 6B depicts an exemplary room presence graphical user interface,according to an embodiment.

FIG. 7A depicts an exemplary lighting control graphical user interface,according to an embodiment.

FIG. 7B depicts an exemplary lighting control graphical user interface,according to an embodiment.

FIG. 8 depicts an exemplary computer-implemented method for performingbuilding automation, according to an embodiment.

FIG. 9 depicts an exemplary computer-implemented method for dispatchingavailable parking spaces to multiple users using an available parkingspaces queue, according to an embodiment.

FIG. 10 depicts an exemplary spot dispatch graphical user interfaceincluding an application.

The figures depict preferred embodiments for purposes of illustrationonly. One skilled in the art will readily recognize from the followingdiscussion that alternative embodiments of the systems and methodsillustrated herein may be employed without departing from the principlesof the invention described herein.

DETAILED DESCRIPTION

Overview

The present techniques provide methods and systems for, inter alia,managing aspects of buildings (e.g., an office building). For example,the present techniques include aspects of building management such asvisitor registration/reception, security badging, parking spotidentification/allocation (e.g., for an electric vehicle), room presencetechniques, lighting control techniques, and administration andreporting techniques. The present techniques improve upon conventionalbuildings by adding automated technological capabilities to enhancebuilding management.

It should be appreciated that the present techniques may be used tomanage one or more physical building structure simultaneously, whetherthe structures are physically joined or not. For example, a hotel mayhave a plurality of locations including hotel rooms, a lobby, a bar, apool, and several conference rooms. For example, the conference roomsmay be detached from the lobby in one or more separate structures. Insuch cases, for purposes of the present disclosure, the “building” mayrefer to the collection locations, whether strictly part of the samephysical structure or not. In some embodiments, disparate physicalstructures (e.g., two data centers in different states) may beconsidered part of the same building management embodiment, wherein thedisparate structures are linked by a computer network.

The present techniques may utilize machine learning (ML) technologies,computer vision (CV) and/or artificial intelligence (AI) to create amodern automated workplace. For example, in some embodiments, thepresent techniques provide automated visitor check-in, notifications,three-factor door security, electric vehicle (EV) parking spot locatingand dispatch, conference room light controls, conference roomdelegation, and reporting in one or more graphical user interface (GUI).Some embodiments include a bot interface and/or backend module.

The present techniques leverage modern technology to automate mundanetasks associated with operating an office building. For example, thepresent techniques eliminate the need for building personnel to turnlights off at night, to adjust heating and cooling, and/or to ensurethat doors are locked. Further, a receptionist is no longer needed tocheck in a visitor or to locate an available conference room. Thepresent techniques may determine the status of parking spaces in realtime and dispatch spaces to users. By automating building managementtasks, building owners will see significant cost and resource savings,building occupants may locate an EV parking spot or finding a conferenceroom, and visitors may seamlessly enter the building environment.Building security is significantly improved by adding multi-factorauthentication that may be configured to use more or fewer factorsdepending on the desired security level of the room or area. The presenttechniques enable visitor badge management and automated kiosk-basedcheck in, and automatic rule-based conference room allocation.

The present techniques improve over conventional chat platforms, whichlack any building automation functionality, by providing users with richfunctionality for accessing and/or controlling important aspects ofbuilding management (e.g., access to office information, control of roomlighting, etc.) using an interface (e.g., a chat bot) that users arealready familiar with. Additional benefits over conventional techniquesinclude simple to use interfaces that provide quick and easy access tobuilding information for building administrators, employees andcustomers. By automating tasks, the present techniques provide a uniformuser experience while improving consistency and reducing errors. Thepresent techniques reimagine how important aspects of office buildingsare managed using modern technologies.

Exemplary Computing Environment

FIG. 1 depicts an exemplary computing environment 100 in which thetechniques disclosed herein may be implemented, according to anembodiment. The environment 100 includes a client computing device 102,a server 104, and a network 106. Some embodiments may include aplurality of client computing devices 102 and/or a plurality of servers104.

The client computing device 102 may be an individual server, a group(e.g., cluster) of multiple servers, or another suitable type ofcomputing device or system (e.g., a collection of computing resources).For example, the client computing device 102 may be any suitablecomputing device (e.g., a server, a mobile computing device, a smartphone, a tablet, a laptop, a wearable device, etc.). In someembodiments, one or more components of the computing device 102 may beembodied by one or more virtual instances (e.g., a cloud-basedvirtualization service). In such cases, one or more client computingdevice 102 may be included in a remote data center (e.g., a cloudcomputing environment, a public cloud, a private cloud, etc.).

In some embodiments, as discussed with respect to FIG. 4C, the computingenvironment 100 may include one or more embedded computing devices(e.g., one or more Raspberry Pi single-board computers). For example, anembedded computing device may be installed in a security system, such asa three-factor authentication system. The environment 100 may includemore or fewer components. For example, in some embodiments, theenvironment 100 may include one or more smart light bulbs, one or moreswitches, one or more power supplies, one or more LED screens, one ormore cameras, one or more EV charging stations, etc.

The network 106 may be a single communication network, or may includemultiple communication networks of one or more types (e.g., one or morewired and/or wireless local area networks (LANs), and/or one or morewired and/or wireless wide area networks (WANs) such as the Internet).The network 106 may enable bidirectional communication between theclient computing device 102 and the server 104, and/or between multipleclient computing devices 102, for example.

The client computing device 102 includes a processor 110 and a networkinterface controller (NIC) 112. The processor 110 may include anysuitable number of processors and/or processor types, such as CPUs andone or more graphics processing units (GPUs). Generally, the processor110 is configured to execute software instructions stored in a memory114. The memory 114 may include one or more persistent memories (e.g., ahard drive/solid state memory) and stores one or more set of computerexecutable instructions/modules 120, including a graphical userinterface (GUI) module 122, and a bot client module 124.

The client computing device 102 further includes an input device 140 andan output device 142. The input device 140 may include any suitabledevice or devices for receiving input, such as one or more microphone,one or more camera, a hardware keyboard, a hardware mouse, a capacitivetouch screen, etc. The output device 142 may include any suitable devicefor conveying output, such as a hardware speaker, a computer monitor, atouch screen, etc. In some cases, the input device 140 and the outputdevice 142 may be integrated into a single device, such as a touchscreen device that accepts user input and displays output. The clientcomputing device 102 may be associated with (e.g., owned/operated by) acompany that services enterprise customers. In some embodiments, one ormore client computing devices 102 may be implemented as a computingkiosk. For example, a building lobby may include one or more kioskcomputing devices including touch screens for use by visitors during ane-reception process.

The NIC 112 may include any suitable network interface controller(s),such as wired/wireless controllers (e.g., Ethernet controllers), andfacilitate bidirectional/multiplexed networking over the network 106between the client computing device 102 and other components of theenvironment 100 (e.g., another client computing device 102, the server104, an electronic database, etc.).

The one or more modules 120 implement specific functionality. Forexample, in an embodiment, the GUI module 122 includescomputer-executable instructions that, when executed, cause a computerto display information (e.g., a web page, a computing application, data,etc) on a computing screen (e.g., via the output device 142). In anembodiment, the GUI module 122 receives input from the input device 140.For example, the GUI module 122 may receive an image via a camera inputdevice 140. The GUI module 122 may receive typed information via a touchscreen or keyboard input device 140. The GUI module 122 may includeinstructions for storing input (e.g., in the memory 120). The GUI module122 may include instructions for transmitting the input via the network106.

The bot client module 124 may store instructions for accessing a remotebot. The remote bot may accept commands and perform other functions, asdiscussed below. The bot client module 124 may convert user input intobot commands that the remote bot is programmed to execute.

The server 104 includes a processor 150 and a network interfacecontroller (NIC) 152. The server 104 may further include a database 190.The database 190 may be a structured query language (SQL) database(e.g., a MySQL database, an Oracle database, etc.) or another type ofdatabase (e.g., a not only SQL (NoSQL) database). The server 104 mayinclude a library of client bindings for accessing the database 190. Insome embodiments, the database 190 is located remote from the server104. For example, the database 190 may be implemented using a RESTdb.IOdatabase, in some embodiments.

The processor 110 may include any suitable number of processors and/orprocessor types, such as CPUs and one or more graphics processing units(GPUs). Generally, the processor 150 is configured to execute softwareinstructions stored in a memory 154. The memory 154 may include one ormore persistent memories (e.g., a hard drive/solid state memory) andstores one or more set of computer executable instructions/modules 160,including a bot module 162, an e-reception module 164, a badging module166, a parking module 168, a room module 170, a lighting module 172, anadmin module 174, a reporting module 176, a machine learning module 178,a queueing module 180 and a scheduling module 182. Each of the modules160 implements specific functionality related to the present techniques.

In some embodiments, a plurality of the modules 160 may implement aparticular technique. For example, the e-reception module 164 and thebadging module 166 may, together, authenticate a visiting user and avisited user. The modules 160 may exchange data via inter-processcommunication (IPC) within a single computing device, such as the server104. In some embodiments a plurality of the modules 160 may beimplemented in a plurality of computing devices (e.g., a plurality ofservers 104). The modules 160 may exchange data among the plurality ofcomputing devices via a network such as the network 106. The one or moremodules 160 FIG. 1 will now be described with respect to a plurality ofGUI applications and exemplary embodiments.

Exemplary Building Management Suite Web Application

The functionality facilitated by the one or more modules 160 may beaccessible via a networked application, in some embodiments. Forexample, FIG. 2A depicts an exemplary building automation GUI 200including an application 202. The application 202 may be a web browser.The browser may correspond to a set of computer-executable instructionsstored in the memory of a client application, such as the clientcomputing device 102 of FIG. 1 . The user may access a menu of availablefunctions 204 that include multiple administrative links. In general,the application 202 is accessed by an administrative user of thecompany. The user may be required to authenticate before the application202 allows the user to access the links within the menu 204. Theapplication 202 may a web application served to the user by, forexample, the admin module 174 of FIG. 1 , via the network 106. Thepresent techniques may provide charts and graphs depicting buildingactivity.

FIG. 2B depicts a graphical user interface 200 including an exemplarybuilding automation bot client application 210, according to oneembodiment. The bot client application 210 may includecomputer-executable instructions for receiving a user input via an inputfield 212, transmitting the user input to a remote bot via a computernetwork (e.g., via the network 106 of FIG. 1 ) and/or for receiving anddisplaying output from the remote bot. In some embodiments, the botclient application 210 includes a plurality of channels 214. Forexample, the depicted embodiment includes a channel 214-A and a channel214-B. In the depicted embodiment, the channel 214-A receives messagesfrom the bot that are of a general building management nature, and thechannel 214-B receives messages related to electronic vehicle parking.Commands typed by the user into the input field 212 may be automaticallyprocessed and analyzed by the remote bot. The input field 212 may acceptcommands that are formatted in a particular grammar, or syntax. Forexample, a command may be provided that includes one or more commandarguments, as discussed below. Output received from the remote bot maybe displayed in an output area 216.

Exemplary Automated Check-In (e-Receptionist) Embodiment

In addition to administrative features, such as the application 202, andbot features, as depicted in the application 210, the present techniquesmay include logging features. For example, FIG. 3A depicts a GUI 300including a table 302 of logging information corresponding to buildingvisitors. Each row of the table 302 includes a visitor full name, acorresponding visited full name, a visitor status, a visitor check intime, and a visitor image. The cells of the table 300 may be links, suchthat a user may access a link within a column (e.g., the status column)to receive/retrieve information or cause an action to occur. Forexample, the user may click a user image to view a larger image. Thee-reception module 162 may generate the table 302 byreceiving/retrieving information from a database (e.g., the database190).

FIG. 3B depicts an exemplary GUI 310 including a login prompt 312. Thee-reception module 162 may display the GUI 310 and the login prompt 312in a kiosk computer of a building lobby, for example. In response to auser accessing the login prompt 312, the e-reception module 162 maydisplay a visitor registration GUI 320, as shown in FIG. 3C. The visitorregistration GUI 320 includes user registration information 322, actionbuttons 324, and a photo 326. The user may enter registrationinformation 322 via an input device (e.g., the input device 140 of FIG.1 ). The user may access the action buttons 324 to cause a camera tocapture and/or recapture the image 326 of the user. For example, FIG. 3Ddepicts an exemplary visitor registration GUI 330, that may correspondto the visitor registration GUI 320.

In the visitor registration GUI 330, the user has entered informationand retaken the photo 326. In particular, the user has entered the nameof a visited person. In FIG. 3E, a visitor registration GUI 340 displaysthe visitor's entered information, along with two photos captured by thevisitor. The visitor registration GUI 340 further includes a progressbar 328 that may be displayed while the visitor registration GUI 340processes the input provided by the user.

Specifically, in some embodiments, the GUI 310, the GUI 320, the GUI 330and/or the GUI 340 are generated by the GUI module 122 of FIG. 1 .Further, each may include instructions for submitting information viathe network 106 to the reception module 164 of FIG. 1 . The receptionmodule 164 may include instructions for processing the input. Forexample, the reception module 164 may store the input in the database190, pass the data to another module (e.g., the bot module 162), or takeanother action (e.g., transmit a notification to the visited person).

FIG. 3F depicts an exemplary GUI 350 for displaying notifications to avisited user. The GUI 350 includes an application 352 that maycorrespond to the bot client application 210 of FIG. 2B, in someembodiments. The application 352 includes an output area 354 and achannel list 356. The application 352 may receive one or more messages(e.g., a notification) in one or more channels concurrently. Forexample, the application 352 may include a plurality of concurrent HTTPconnections to a remote server (e.g., the server 104). The application352 may include instructions for displaying visual feedback (e.g., atext color, an animation, etc.) in association with display of thechannels. The visual feedback may indicate when a message is receivedinto one of the respective channels.

In some embodiments, the application 352 may display the one or messagesin the output area 354. For example, as depicted in FIG. 3F, theapplication 352 may display a visitor arrival notification 358. Thevisitor arrival notification 358 may include information correspondingto that entered by the visitor (e.g., into the visitor registration GUI320), wherein the information identifies the visitor to the user of theGUI 350 (i.e., the person visited). The visitor arrival notification 358may further include one or more response inputs 360, wherein the usermay access the response inputs 360 to transmit a predetermined messageto the visitor and/or to view additional information about the visitor.

The GUI 350 may include instructions that are executed (e.g., by the GUImodule 122 of FIG. 1 ) in response to the user accessing one or more ofthe response inputs 360. For example, the user may access a five minutesout button of the response inputs 360. In response to the user's access,the GUI module 122 may cause a message request to be transmitted to anotifications module in the server 104 (not depicted). The messagerequest may include the information of the visitor, including thevisitor email address, as well as a message corresponding to theaccessed button or other GUI element. The notifications module may causea notification to be transmitted to the visitor.

FIG. 3G depicts an exemplary notification GUI 370 including an emailmessage 372 corresponding to a notification. The email message 372includes a message notifying the visitor corresponding to the responseinput 360 accessed by the visited user via the GUI 350. In someembodiments, other modes of communicating the notification may be used(e.g., a push message, a text message, an automated telephone call,etc.). It should also be appreciated that the response inputs 360 mayinclude other predetermined notification messages and/or more complexfunctionality (e.g., the user may be prompted to enter a custom messageto send to the visitor). In some embodiments, the user may reject thevisitor. In still further embodiments, the user may enter bot commandsinto an input field 362. For example, the user may enter a command suchas “notify visitor meet at the coffee shop.” In that case, the GUI 370may include instructions for analyzing the user's bot command (e.g., viathe bot client module 124 of FIG. 1 ). The bot client module 124 maydetermine that “notify visitor” corresponds to a notification command,and that the remaining bot command (i.e., “meet at the coffee shop”) isan argument for transmission as a message payload to the visiting user.The “notify” command may be sent by a preconfigured default notificationroute. In some embodiments, the user may specify a notification route(e.g., “email visitor,” “text visitor,” etc.).

FIG. 3H depicts a GUI 380 including a table 382 of logging informationcorresponding to signed in building visitors. Continuing the previousexample, the visited user may greet the visitor (e.g., at a buildinglobby). The visited user may use one or more credentials (e.g., apersonal identification number (PIN), a biometric credential, a radiofrequency identification (RFID) tag, etc.) to identify the visited user.The visited user may self-identify by entering the one or morecredentials, for example, into a client computing device (e.g., acomputing kiosk) and authorizing the visitor based on the personalrecognizance of the visited user. In some embodiments, the visited usermay enter personally-identifiable information corresponding to thevisitor into the client computing device (e.g., a scan of the visitor'sstate-issued identification card).

Once the visited user has authorized the visiting user, the clientcomputing device (e.g., the client computing device 104) may transmit arequest including the identification of the visitor and the identity ofthe authorizing visited user to a remote computing device (e.g., theserver 104). The server 104 may generate a timestamp and insert a signin record into the table 382. The sign in records provide administratorswith an audit trail that advantageously allows the present techniques toautomatically track the entry of visitors. The table may be, forexample, a table stored in the database 190 of FIG. 1 . In someembodiments, the reception module 164 of the server 104 may receive therequest transmitted by the client computing device 102. The receptionmodule 164 may cause another module (e.g., the badging module 166) totransmit a response to the client computing device 102. The response mayinclude an authorization token and/or message notifying the visited userthat the visitor may enter the premises of the building. The responsemay cause a turnstile or other physical barrier to open, allowing thevisitor and/or visited user to physically access the premises of thebuilding.

In still further embodiments, the response may cause a device (e.g., a3D printer, an RFID card generator, etc.) to print and dispense aphysical token for the visitor. The token may allow building occupants(e.g., an employee, a security officer, etc.) to visually identify thevisitor as an authorized guest. The token may be automatically analyzed(e.g., by an RFID reader) to determine the location of the visitorwithin the building, and to determine when the visitor exits thebuilding. Such RFID readers may cause the information in the table 382to be updated (e.g., by inserting a sign out time). RFID tags and otherforms of authentication data may be used in some embodiments to securethe premises of the building.

Exemplary Security Badging Embodiment

Security badging may be used to track the movements of visitors andbuilding occupants within the building. For example, FIG. 4A depicts aGUI 400 including a monitoring application 402. The monitoringapplication 402 includes an output area 404 and an input field 406. Thebadging module 166 may monitor one or more aspects of the building viaone or more sensors (e.g., via an RFID sensor as discussed above, via acamera, etc.). The badging module 166 may include a machine learningmodel trained to detect one or both of (i) motion, and (ii) objects(e.g., a human). When the badging module 166 detects motion, the badgingmodule 166 may generate a notification and transmit the notification fordisplay in the output area 404. As depicted, the notification mayinclude a message including the category of detected object (e.g.,Person), the location of the object (e.g., Zone 1 Door) and/or a livephotograph or a file photograph. The badging module 166 may identify afile photograph by comparing a live photograph (i.e., a photographcaptured in response to motion detection) to one or more lists ofphotographs. The lists of photographs may be searched in a priorityorder. For example, the badging module 166 may search current visitorphotographs, followed by current employee photographs. In someembodiments, the badging module 166 may be configured to search foremployees followed by consultants/contractors, as denoted by personnelrecords in the database 190.

In some embodiments, the present techniques may implement multi-factorbadging authentication. Specifically, a badging client device may beconfigured to display a user registration GUI, such as the GUI 410 ofFIG. 4B. Such a badging client device may be an instance of the clientdevice 102, for example, and may be located in a secured location toprevent unauthorized access (e.g., in a building security office). Whena new person (e.g., a building occupant/employee) needs access to thebuilding, either temporarily or more permanently, the new person mayenter information into an application 412 of the GUI 410. Theapplication 412 may include instructions for displaying a form tocollect user information, such as the full name of the user. The usermay provide a PIN and may be assigned a physical RFID tag having an RFIDnumber. The application 412 may include further instructions forcollecting a profile photograph (e.g., a photograph of the user's face).Each of the pieces of information collected by the application 412 maybe a respective factor used to identify the user. Some or all of thefactors may be required to successfully identify the user, depending onthe embodiment.

FIG. 4C depicts an exemplary computing environment 420 for implementingmulti-factor authentication. The environment 420 includes a power supply422, a lock relay 424, a door lock 426, a breadboard 428, a single-boardcomputer 430, an LCD display 432, a camera 434, and an RFID tag 436. Thepower supply 422 may be a 12-volt power supply that operates in AC/DCmode, and supplies electric power to the other components of thecomputing environment 420. The lock relay may respond to electricalsignals from the breadboard 428 and lock or unlock a locking rod orlatch within the door lock 426 to an open or closed position. Thebreadboard 428 may be a general purpose input/output (GPIO) board, insome embodiments. The RFID tag 436 may be accessible via USB in someembodiments. The door lock 426 may include an electric door strike, insome embodiments.

The single-board computer 430 may be, for example, a Raspberry Picomputer. The single-board computer 430 may correspond to the clientcomputing device 102, in some embodiments, and may include instructionsfor reading a PIN of a user and/or for reading information from the RFIDtag 436. The single-board computer 430 may include instructions forcapturing one or more images via the camera 434. The camera 434 may beany suitable wired and/or wireless (e.g., networked) camera, and maycapture images in the visible spectrum and/or infrared images.

In an embodiment, the server 104 may implement the single-board computer430. In the case, the badging module 166 may include instructions forcapturing a PIN, a photograph of a person, and/or an RFID tag/token(e.g., and RFID tag corresponding to the RFID tag 436). As depicted inFIG. 4D, to authenticate a user, the badging module 166 may request(e.g., via a GUI 440) that the user present the RFID tag. The badgingmodule 166 may read a unique identifier (UID) from the RFID tag, whichmay act in a passive or active mode. The badging module 166 may comparethe UID to a list of stored UIDs (e.g., by querying the database 190).When the UID matches one or more of the stored UIDs, the badging module166 may return a Boolean value corresponding to a match or no match. TheUID search may be a binary search, in some embodiments.

In some embodiments, once the badging module 166 determines whether theUID matches a stored UID, the badging module 166 may proceed to analyzea second factor. For example, in some embodiments, the badging modulereceives/retrieves an identifying factor from the user (e.g., a PIN).For example FIG. 4E depicts an exemplary GUI 450 including a PINapplication 452 for receiving/retrieving a PIN from the user. In someembodiments, the application 452 corresponds to a set ofcomputer-executable instructions stored in one of the modules 120, suchas the GUI module 122. In some embodiments, the application 452 requestsa PIN in response to the user's successful supply of a first factor(e.g., an RFID matching a stored RFID corresponding to the user). Insome embodiments, the application 452 may display the number of digitsentered by the user, as depicted in FIG. 4F.

The badging module 166 may collect a third factor from the user in someembodiments. For example, once the user has supplied a known RFID tagand a PIN corresponding to the known RFID tag, the badging module 166may request a photograph of the user, as depicted in GUI 460 of FIG. 4G.

The GUI 460 may include an application 462 that instructs the user tolook at a camera. For example, the client 102 may display theapplication 462 in the output device 142 of the client 102. Theapplication 462 may display an image capture countdown. A cameraaccessible to the client 102 (e.g., the input device 140) may capture aphotograph of the user and display a message to that effect in theoutput device 142.

The client 102 may transmit the captured photograph of the user and anindication of the user's identity (e.g., a username, email address, orother unique identifier) to a remote computing device for analysis, suchas the server 104. A module of the server 104 (e.g., the ML module 178)may retrieve a stored photo of the user using the unique identifier, andcompare the captured photograph of the user to the stored photograph topredict a likelihood that the captured photograph and the storedphotograph are of the same person. In some embodiments, the ML module178 may analyze the photograph of the user using a trained facialrecognition model. When the likelihood is above a predeterminedthreshold, the ML module 178 may indicate that the photos are a match.

The match of the two photos may be used as a third authenticating factorin some embodiments. In such embodiments, the successfully-authenticateduser will have provided 1) physical access to a device provided by thebuilding management (e.g., the RFID tag); 2) information known only tothe user (e.g., the PIN); and 3) information indicating the physicalidentity of the user (e.g., the photograph). Thus, the presenttechniques provide a very strong authentication guarantee, in that theuser has provided multiple factors. An attacker may be able to forge orsteal one, or possibly two of the factors, but achieving simultaneousforgery/theft of all three factors may be much more difficult.

It should be appreciated that embodiments are envisioned whereinmultiple instances of the computing environment 420 may be implementedin a building. For example, a first computing environment 420 maycontrol a first door lock, while a second computing environment 420controls a turnstile. The modules 120 and/or modules 160 may includeinstructions for managing the status of multiple independent instancesof the computing environment 420.

Exemplary Electric Vehicle Spot Finder Embodiment

FIG. 5A depicts an exemplary help menu GUI 500 including a buildingmanagement application 502. The application 502 includes an input field504 and an output area 506 that may correspond, respectively, to theinput field 406 and the output area 404 of FIG. 4A, for example. Theinput field 504 may accept one or more input commands and/or inputparameters. The input commands and/or input parameters may be receivedby a set of computer-executable instructions executing in a memory, suchas the GUI module 122 of FIG. 1 . For example, in the depicted exampleof FIG. 5A, the user has typed the command 508. The command 508 isreceived by the GUI module 122, in an embodiment. The GUI module 122 mayprocess the command, and output a message 510 into the output area 506.It should be appreciated that in the depicted example, the user isrequesting help.

The help output may be stored as a static message, and therefore, theGUI module 122 may display the help output in the output area 506 byretrieving the static message from a memory (e.g., the memory 113, thedatabase 190, etc.). However, for some commands that include dynamicoutput, and/or output that requires further analysis/computation, theGUI module 122 may cause one or more modules of a client computingdevice (e.g., the computing device 102) to process the user command togenerate the output, and/or transmit one or more requests, and/orreceive one or more responses to such transmitted requests via a networkand/or a remote computing device (e.g., by accessing the server 104 overthe network 106). For example, the bot client module 124 may processcommands and/or parameters entered by the user.

The message 510 depicts exemplary commands available to the user (e.g.,a building automation manager, an employee, etc.). In the depictedembodiment, the commands may be configured to manage aspects of electricvehicle parking, as shown in further detail in FIG. 5B.

FIG. 5B depicts an exemplary GUI 520 that may correspond to the GUI 500.The GUI 520 may include an application 522 that has an input field 524and an output area 526. The application 522 includes a user command(e.g., find EV locations) parameterized by a distance (e.g., 4 miles).The command output 530 includes a list of locations within a radius ofthe distance, including descriptive information about each of thelocations. The descriptive information may include a name, an address, arespective list of port types, and a link allowing the user to access amap of the location. When the user accesses the link, the application522 may be configured to display map, as depicted in FIG. 5C.

FIG. 5C depicts a GUI 530 including a map 532. The map 532 includes amap description 534 corresponding to one of the list of locationsdepicted in FIG. 5B. The map includes a route 536 depicting a route fromthe user's current location to the one of the list of locations.Therefore, by using commands, the user may find suitable parkinglocations that are nearby the user's current location. In someembodiments, the user's current location is fixed, such as when the useris within a building. In some embodiments, the GUI module 122 may accessa GPS module of the client 102 (not depicted) to determine the user'slocation. In still further embodiments, web-based technologies (e.g.,Geolocation) may be used to determine the location of the user based onthe location of the user's device.

The present techniques may be used to identify an open/available EVparking spot/space. For example, the user may enter acommand/instruction into the input field (e.g., “Check for open spaces”)as shown in FIG. 5A. The GUI module 122 may be configured to analyze animage of a physical area to identify one or more open space. Forexample, FIG. 5E depicts a parking area 550 including one or moreparking spaces each of which may contain a respective vehicle. Theparking area 550 may correspond to a photograph of a parking lot, insome embodiments. In some embodiments, the GUI module 122 may transmitthe user command to a module of a remote computing device, such as theparking module 168 of FIG. 1 . The parking module 168 may furtheranalyze the command, for example, via the ML module 178.

The parking spaces in the parking area 550 may be marked or unmarked,wherein the presence of a marking indicates a spot type. For example,the parking spaces may include one or more EV parking spaces 552,denoted by the presence of an EV marking. The marking may be applied tothe parking space using paint, for example. An accessible, disabled orhandicap parking space may include another marking type. The EV parkingspaces may each include a charging port 554.

In some embodiments, the charging port 554 may be an instance of theclient 104. For example, the charging port 554 may be communicativelycoupled to the network 106. The charging port may include an inputdevice 140 and an output device 142, for example. The charging port 554may provide electric power to EVs for charging, and may include a datalink communication to/from EVs.

As noted, the present techniques may assist a user by identifying anopen/available EV parking space. Identifying the open/available EVparking spaces may include the application of computer vision techniquesand/or machine learning. For example, the ML module 178 of FIG. 1 may beconfigured to analyze an image of a physical area (e.g., the parkingarea 550) using one or more trained ML model. The image of the physicalarea may be captured by a camera communicatively coupled to the client102 or the server 104, for example. For example, in an embodiment, theclient 104 may be a camera capture device stationed in a parking area(e.g., the parking area 550).

The camera may correspond to the camera 434, in some embodiments. Thecamera may be implemented as a using a Cisco Meraki Cloud ManagedSecurity Cameras, in some embodiments. The camera may be communicativelycoupled to the network 106, and thus, images captured by the camera maybe analyzed by the one or more trained ML model of FIG. 1 . The one ormore ML model may be trained by the ML model 178 analyzing labeledimages. The one or more ML model may be operated by the ML module 178 ofFigure and/or by the parking module 168.

In some embodiments, an existing library/programming language may beused for implementation, such as Python/OpenCV. A pre-trained networksuch as the Resnet 50 neural network may be implemented. GoogleTensorFlow may be used to implement a particular ML technique. In someembodiments, a convolutional neural network (CNN) may be trained on manyimages to recognize/identify vehicles. For example, a pre-trained deepneural network may be used to identify only electric vehicles in theparking area 550. Other supporting technologies that may be used toimplement aspects of the present techniques in some embodiments includexAPI Phillips-E, Amazon Alexa, AWS Rekognition, Dropbox file storageapplication programming interface (API), Google Dialogflow API, MerakiAPI, JavaScript Webcam API NodeMailer email API Open Charge Map API,Phillips HUE API and Cisco Webex Teams API. Further technologies thatmay be used to implement the present techniques include Python,JavaScript, Tkinter, jQuery and Node.js.

For example, in some embodiments, the modules 120 may include a voicerecognition module (not depicted) that integrates with Amazon Alexa toaccept voice commands from the user. The user may speak a command suchas, “Alexa, tell the general room a message.” The user may be promptedto speak a message. The user may respond, “Tell bot to show datacenter.” In response to the user's command, the bot client 124 may becaused to display the requested message on the output device 142.Additional voice-enabled functionality may be implemented using anexisting third-party API and/or using a voice recognition module in themodules 120 (not depicted). The user may query the number of conferencerooms, for example, including the number of occupants. The user maycontrol lighting using the voice-enabled functionality.

The user may enter a command that accesses the EV parking spaceidentifying capabilities of the present techniques. For example, FIG. 5Fdepicts a GUI 570 including an application 572. The application 572includes an input field 574. The application 572 depicts an exampleinput command 576. The input command 576 may be output, or echoed, tothe application 572 via an output device, such as the output device 142.The input command 576 may be received via the GUI module 122 andadditionally/alternatively transmitted to a command processing module,such as the bot client 124 and/or the parking module 168. The parkingmodule 168 may receive/retrieve a real-time image of the parking spacesin the parking area 550. The parking module 168 may analyze the image ofthe physical area to identify one or more vehicles in the parking area550. The parking module 168 may further identify one or more spaces.Identifying the vehicles and spaces may include analyzing the imageusing one or more respective trained ML models.

For example, the parking module 168 may analyze the image using one ormore trained ML models. The ML models may analyze contours of the image.The parking module 168 identify one or more contours of the image andmay iterate over the contours. The parking module 168 may use aDouglas-Peucker algorithm to generate an approximate contour for theimage, wherein the approximation identifies one or more parking spots556 by identifying bounding boxes corresponding to an object of interest(e.g., a parking space, a vehicle, etc.). The one or more ML models maystore a set of pixel coordinates (e.g., x1, y1, x2, y2) defining eachbounding box relative to the image.

In some embodiments, a vehicle detection ML model may identify a list ofvehicles using a vehicle object detection machine learning model. Theparking module 168 may receive a set of pixel coordinates for eachidentified vehicle from the ML model, and store the pixel coordinates(e.g., in the database 190). The vehicle object detection ML model maybe a retina net and may identify vehicles such as passenger cars,motorcycles, and/or trucks, as depicted in FIG. 5E. The output of thevehicle object detection ML model may include a vehicle type flagdenoting the vehicle type as a passenger vehicle, motorcycle, truck,etc. The vehicle object detection ML model may identify the one or morevehicles in the parking area 550 as discussed above by drawing one ormore vehicle bounding box 558 around the identified vehicles in theimage. A spot detection ML model may identify a list of EV spots. Thespot detection ML model may store pixel coordinates for each of theidentified EV spots.

The parking module 168 may compute the area of spot bounding boxes 556and the area of vehicle bounding boxes 558, and compare the respectiveareas, to determine respective overlap. When there is a predeterminedamount of overlap (e.g., less than 40%), the parking module 168 maydetermine that the spot is unavailable. Of course, the overlap thresholdmay be adjusted to increase the sensitivity of the modeling. When thereis not overlap, the parking module 168 may determine that the spot isavailable. The status of the respective parking spots within the parkingarea 550 may be stored in the database 190, for example. The historicalstatus of the parking spots may be stored and queried over time. Theparking module 168 may compute the width and height of the overlap ofeach vehicle bounding box with each spot bounding box by computing theintersection of the boxes.

In some embodiments, as depicted in FIG. 5F, the bot may notify thatuser via the application 572 that no spots are available. In such cases,the user may be prompted to query nearby EV parking locations withparticular port types as shown in FIG. 5D, sorted by least distance, andfiltered by port type. Specifically, the user command 542 includes auser-specified port type 544 and a user-specified proximity 546. In thisway, the present techniques advantageously improve the efficiency oflocating geographically proximate charging stations 548 that are limitedto an appropriate port type for the user's vehicle, and are within aspecified distance. In some embodiments, the charging stations 548 maybe automatically displayed to the user in response to detection by thepresent techniques of a full local parking lot.

The spot identification techniques discussed with respect to FIG. 5E maybe used periodically and/or upon user demand. For example, FIG. 5Gdepicts an exemplary GUI 580. The GUI 580 includes an application 582that may correspond to the building automation bot client application210, in some embodiments. The application 582 may include an input area584 for receiving user commands and an exemplary command 586. Theapplication 582 may include a first exemplary output message 588-Anotifying the user, in response to the input command 586, that a spot isavailable in the parking area 550. The spot identification may functionas discussed with respect to FIG. 5F. In some embodiments, the open spotmay be identified visually by way of color highlighting, font styling(e.g., bold/italicized text), animation or other suitable means.

The spot identification techniques discussed with respect to FIG. 5E maybe periodically executed (e.g., once every five minutes) and when thestatus of the parking area 550 changes, the parking module 168 maygenerate an output and transmit the output to the user. For example, theparking module 168 may periodically call the trained spot identificationmodel. The model may generate output indicating that a spot isavailable, and/or that an available spot is no longer available. Asdepicted in FIG. 5G, the model may generate a second exemplary outputmessage 588-B notifying the user that the spot is no longer available inthe parking area 550.

FIG. 5H depicts a parking space request queue 590. In some embodiments,the parking space request queue 590 may be implemented as a first-infirst-out (FIFO) queue. The parking space request queue 590 may includea number n of queue elements 592. The parking space request queue 590may include a head element 592-B and a tail element 592-E. The parkingspace request queue 590 may support queue elements, such as removing thehead element (dequeueing), as shown in FIG. 5H by the removal of anelement 592-A. The parking space request queue 590 may support addingthe tail element (enqueueing), as shown in FIG. 5H by the adding of atail element 592-F.

The parking space request queue 590 may be ordered as depicted in FIG.5H, such that the element 592-A is the first added and first removed,the element 592-B is the second added and second removed, an element592-C is the third added and third removed, the element 592-E is the nthadded and nth removed, and an element 592-F is the n+1th added and n+1thremoved. In other words, in a FIFO queue embodiment, the elements 592may be added and removed in the same order. Each of the parking spacerequest queue 590 elements may correspond to an input command, such asthe input command 576. For example, multiple users may request an openspace by entering respective input commands 576 into multiple instancesof the client computing device 102. The parking space request queue 590may support instructions for counting the number of elements in thequeue, and for searching the queue, for example to find the indexvalue/position of an element (e.g., a user command) within the queue.

In some embodiments, the parking space request queue 590 may orderavailable parking space requests from multiple users and avoid assigninga single open parking space to multiple requesting users. Specifically,in some embodiments, multiple users may request an available space. TheGUI module 122 of each respective user device 102 may transmit each userspace request to the queueing module 180. The queuing module 180 mayinsert each user space request into a parking space request queue 590.In such cases, the parking module 168 may include instructions forchecking the size of the parking space request queue 590. When theparking space request queue 590 is non-empty, the parking space module168 may analyze a parking area (e.g., the parking area of FIG. 5E) asdiscussed herein to identify one or more open spaces. For eachidentified open space returned by a trained ML model, the parking module168 may dequeue one of the requests from the parking space request queue590 and notify the user corresponding to the request (e.g., via thenetwork 106) that the respective identified space is assigned to theuser. The parking module 168 may include a parking space identifier. Thespace identifier may be included in each space (e.g., using a paint, asign, a label, etc.), so that the user can identify the correct spacefor parking. The space identifier is noted in FIG. 10 , below.

In some embodiments, the parking space module 168 may request openspaces from the trained ML module on demand in response to a userrequest. In other embodiments, the scheduling module 170 mayperiodically request open spaces from the ML module 178. For example,the scheduling module 170 may request open spaces at an interval (e.g.,every 5 minutes) and assign the open spaces by dequeueing any requestsfrom the parking space request queue 590. Users may access the parkingspace request queue 590 using a client computing device 102. Such usersmay be building administrators, building employees, employees ofbusinesses located within the building, and/or building visitors.

In some embodiments, the parking module 168 may instantiate multipleparking space request queues 590. For example, the parking module 168may instantiate a first parking space request queue 590 to handlegeneral parking requests (e.g., for non-EV parking). The parking module168 may instantiate a second parking space request queue 590 to handleall EV requests. The parking module 168 may instantiate a third parkingspace request queue 590 to identify spaces having a specific chargingport type. Practically, any suitable number of individual parking spacerequest queues 590 may be supported. For example, each tenant of a largebuilding (e.g., a building housing tens of thousands of occupants ormore) may be assigned one or more parking space request queue 590. Eachparking space request queue 590 may be associated with particularparking spaces. Each parking space request queue 590 may be associatedwith an individual trained ML model that is trained, respectively, toidentify spots having particular characteristics (e.g., owned by thetenant, supporting J1772 charging, non-EV parking space, etc.).

Exemplary Room Presence Embodiment

In an embodiment, the present techniques may be used to manage aspectsof room presence. For example, FIG. 6A depicts a GUI 600 including atable 602 of room presence information. The table 602 may include one ormore rows, each corresponding to a room within the building/buildings.Each row may include room information such as a name, a Boolean flagindicating the presence of people in the room, a count of people, a callstatus, a map link, etc. A user may access information corresponding toeach room by accessing the map link. The GUI 600 may includeinstructions that cause the GUI 600 to display additional information inresponse to user input. For example, when the user accesses the map linkwithin a row of the table 602, the GUI 600 may display a detailed map ofthe room corresponding to the accessed row.

FIG. 6B depicts a room layout GUI 610 that includes a detailed real-timeroom view 612. The room view 612 includes a room diagram 614 including aperson indicator 616. When the user accesses the person indicator 616,the GUI 610 may display a person identifier 618 that includes theidentity of the corresponding person. The GUI 610 may receive the nameof the room and/or a unique room identifier as a parameter when the useraccesses the map link of the GUI 600. The GUI 610 may includeinstructions for querying the database 190, for example, to retrieve alist of names of people whose RFID tags are located within the roomdepicted in the room view 612. In some embodiments, the room module 170may include computer-executable instructions that when executed, analyzethe room presence information to find a next-available conference room.

In some embodiments, the presence of a person in a room may be detectedby the room module 170, for example, detecting activity in the room. Forexample, the room module 170 may continuously monitor one or more videofeeds each corresponding to a room. The room module 170 may includeinstructions for processing individual video frames to detect an object(e.g., a person).

Exemplary Lighting Control Embodiment

The present techniques may be used to control aspects of room lighting.For example, FIG. 7A depicts an exemplary GUI 700 including lightingcontrol access 700. The user may access the lighting control access 700,via a secured control panel that requires authentication. For example,the control panel may correspond to the input device 140, and thelighting control access may be provided in a stationary access panelimplemented as an instance of the client 104 (e.g., a panel affixed to awall within the building). In some embodiments, the lighting control maybe implemented as an application installed in a smart phone of a user.The rooms may include one or more occupancy sensors that allow thelighting control module 172 to determine and/or count the presence ofpeople within the respective rooms.

The user may control the status of lighting within the building usingthe lighting control. For example, FIG. 7B depicts a lighting controlGUI 720 that includes a light control application 724. The lightingcontrol application 724 may include one or more lighting control buttonsthat when accessed by the user (e.g., via a mouse click, a touch screenpanel, etc.) cause one or more electric light within a respective roomto toggle on or off. For example, the lighting control buttons may eachcorrespond to one of the rooms within the room presence informationtable 602. The lighting control buttons may include a graphicalrepresentation of a respective status. For example, a lighting controlbutton may be illuminated or colored when turned on, and notilluminated/colored when turned off.

In some embodiments, the lighting within the rooms may be implementedusing smart lighting (e.g., Phillips Hue lighting). The lightingcontrols may be integrated into the bot system described above, suchthat in some embodiments, the bot client module 124 receives a usercommand (e.g., “turn on lights in SpaceX Room”), analyzes the command toidentify that the command includes an instruction (“turn on lights”) anda parameter (“SpaceX Room”). The bot client module 124 may transmit theinstruction and parameter to the lighting module 172 of the server 104.The lighting module 172 may include a set of computer-executableinstructions for controlling lighting, and may perform an action to aparticular set of lights based on the instruction and the parameter. Thebot client module 124 may receive a response message from the lightingmodule 172 and display the response message to the user.

Exemplary Methods

FIG. 8 depicts a flow diagram of an example method 800 for classifying avehicle parking space. The method 800 may include receiving, in an inputfield of an application of a graphical user interface, a user commandincluding an instruction to locate an available vehicle parking space(block 802). For example, the user command may be entered by a user intothe input device 140 of FIG. 1 . The GUI module 122 may includeinstructions for receiving the user command. The GUI module 122 may passthe user command to the bot module 124. The bot module 124 may processthe user command to extract one or more instructions including one ormore parameters. The bot module 124 may transmit the user command viathe network 106 to the bot module 162. The bot module 162 may transmit aresponse to the user for display in the output device 142 by the GUImodule 122. The application may be a building management botapplication, such as the application 202 depicted in FIG. 2A. The inputfield of the application may correspond to the input field 574 of FIG.5F, for example. The user may type a command such as the command 576,for example.

Concurrently, the bot module 124 may analyze the user command todetermine the user's intent. A trained intent processing ML model may beused in some embodiments to determine the user intent. Based on theuser's intent and/or the contents of the user command, the bot module162 may access one or more of the modules 160. For example, the botmodule 162 may pass the user command and/or one or more arguments of theuser command to the ML module 178 for processing. For example, when theuser command relates to finding an available vehicle parking space, theML module 178 may analyze an image of a parking area using one or moretrained ML model to identify one or more available parking spaces.

Specifically, the method 800 may include analyzing an image of a parkingarea (e.g., the parking area 550 of FIG. 5E) using a spot detectionmachine learning model to identify one or more parking space boundingboxes, each corresponding to a respective set of pixels (block 804). Insome embodiments, the spot detection machine learning model includesanalyzing the image of the parking area to identify one or more contoursrelating to a painted region of the parking space.

The method 800 may further include analyzing the image of the parkingarea using a vehicle detection machine learning model to identify one ormore vehicle bounding boxes, each corresponding to a respective set ofpixels (block 806). In some embodiments, the vehicle detection machinelearning model may include analyzing the image of the parking area usinga retina net configured to identify a vehicle type. The method 800 mayinclude comparing each identified parking space bounding box to eachvehicle bounding box by calculating the area of intersection of eachparking space bounding box to each vehicle bounding box (block 808).

The method 800 may include, when the area of the intersection does notexceed a predetermined percentage value, classifying the parking spacecorresponding to the parking space bounding box as available (block810). For example, the predetermined percentage value is 40%. In someembodiments, the predetermined percentage value may be determined byiteratively testing multiple percentage values to minimize a loss value.In some embodiments, the percentage value 40% may be determined using aseparate ML model. The ML model may output an integer value from 0 tothe number of spaces in the parking area, reflecting the number ofavailable spaces. The ML model may also output an image of each space,in some embodiments, as depicted in exemplary output message 588-A andexemplary output message 588-B of FIG. 5G.

It should be appreciated that in some embodiments, a single ML model maybe trained to locate parking spaces and/or vehicles. In someembodiments, other types of ML models may be used to identify availableparking spaces. The ML module 178 may be used to train multiple models,wherein the respective models are trained to identify EV parking spaces,non-EV parking spaces, all parking spaces, etc.

Once the ML module 178 has analyzed the user command, the bot module 162may receive output of the ML module 178. The bot module 162 may transmita response to the user. For example, the method 800 may includetransmitting a notification to a client computing device of a user, thenotification including an identification of the available parking space,and displaying the notification in an output area of a bot graphicaluser interface of the client computing device, as depicted in FIG. 5G.It should be appreciated that the method 800 may include transmitting anotification that the lot is or has become full, as depictedrespectively in FIG. 5F and FIG. 5G. As discussed herein, the results ofthe ML module may be stored in the database 190. Therefore, the statusof the parking space may be tracked over time, and a first and secondnotification may refer to a single spot. For example, a firstnotification may identify an available space, and a second notificationmay identify that the space has become full. When the output of the MLmodule 178 indicates no available parking spaces, the method 800 mayinclude displaying a list of nearby parking locations, such as thecommand output 530 of FIG. 5B.

FIG. 9 depicts an exemplary method 900 for performing vehicle spotdispatch. The method 900 may include receiving a plurality of usercommands each including a respective instruction to locate an availableparking space (block 902). The method 900 may include enqueueing eachuser command in a parking space request queue (block 904). The method900 may include analyzing an image of a parking area using two or moremachine learning models to identify one or more available parking spaces(block 906). The method 900 may include, for each of the one or moreidentified available parking spaces: dequeueing a user command from theparking space request queue, and notifying the user corresponding to theuser command of an identification of one of the identified availableparking spaces (block 908).

In some embodiments, the method 900 may include for each of the usercommands in the parking space request queue, automatically removing theuser command when the user command is expired. In some embodiments, theparking space request queue is implemented as a FIFO queue. In someembodiments, the method 900 includes receiving user commands having arespective future time period (e.g., 2 PM tomorrow), and furtherincludes updating an electronic calendar object based on each respectivefuture time period. For example, the method 900 may reserve a parkingspot of a user at the future time, as discussed herein.

The method 900 may include selecting the parking space request queuefrom a plurality of parking space request queues based on an attributeof the user. For example, the queue may be chosen based on theidentity/affiliation of the user, the type of vehicle, the contents ofthe user command, an intent of the user detected using a trained MLmodel, etc.

The method 900 may include searching parking space request queue to findthe position of the user command corresponding to the user, andtransmitting a notification to the user of the user's position in theparking space request queue. Transmitting the position of the user maynotify the user that there are a number of people in the queue 590 aheadof the user, as shown in FIG. 10 . In some embodiments, the user mayhave the option to submit a “cancel” command, removing the user commandfrom the queue. The method 900 may include periodically analyzing theimage of the parking area using the two or more machine learning modelsto identify the one or more available parking spaces. In this way, thepresent techniques support both on-demand available parking spacedispatch, queued parking space dispatch and/or scheduled parking spacedispatch.

Exemplary Vehicle Spot Dispatch Embodiment

FIG. 10 depicts an exemplary spot dispatch GUI 1000 including anapplication 1002. The application 1002 includes an input field 1004, andoutput area 1006 for displaying responses, user inputs 1008, and programoutputs 1010. User input 1008-A corresponds to a user request to reservea parking space. The user request may be received by the GUI 122 of theclient 102, for example. As noted above, multiple client devices 102 maybe operated simultaneously by multiple users (e.g., by 1024 or moreconcurrent users). The GUI 122 may transmit the user input 1008-A in anetwork request via the network 106 to the parking module 168, forexample. The network request may include a user identifier and othermetadata (e.g., a request time, a vehicle type/identifier, a tenantidentifier, etc.). The parking module 168 may analyze the request andpass the request including one or more metadata element to the queueingmodule 180. The queueing module 180 may enqueue the request in one ofthe parking space request queues 590 based on the metadata. For example,when the request includes an EV vehicle type, the parking module 168 maypass the request to an EV parking space request queue 590.

The parking module 168 may include a queue item expiry in someembodiments. The queue item expiry may be a predetermined time value(e.g., a number of minutes) beyond which queue items expire. Forexample, the queueing module 180 may periodically analyze each elementfrom the tail of the queue until an unexpired element is identified. Foreach expired element, the queueing module 180 may cause a notificationto be transmitted to the user indicating that a spot could not be found.In some embodiments, the user command may include a queue item expiryoverride, such that the default expiry is replaced by the user-specifiedexpiry. In this way, the user may indicate a willingness to wait longerthan the default, or a need to find a parking space sooner.

In some embodiments, the present techniques support rich schedulingfacilities. For example, the scheduling module 182 may maintain one ormore electronic calendar objects (e.g., in the database 190). The GUImodule 122 may include instructions for displaying a calendar in theoutput device 142. The user may access the calendar to select parkingspaces in the future. An individual calendar may be displayed for eachof a number of buildings managed by the environment 100. The user maycreate calendar entries corresponding to desired parking dates andtimes. For example, the user may create a calendar entry via the GUI 122corresponding to parking at a particular building from 10:00 AM-11:00 AMon Tuesday. The user may create a calendar entry that recurs (e.g., onceper week or at another interval).

In some embodiments, the GUI 122 may include instructions for acceptinguser calendar entry requests. In such embodiments, the GUI 122 maydisplay the user's calendar entry using a style indicating that therequest is not confirmed. The scheduling module 182 may receive entryrequests including the metadata discussed above from the GUI 122 andanalyze the one or more electronic calendar objects to determine whetherto accommodate the user calendar entry request. The scheduling module182 may include a set of computer-executable rules for approving usercalendar entry requests. For example, a rule may specify that a user mayclaim one hour of parking per calendar day, but no more. Another rule,for example, may specify that a user may not park a non-electric vehiclein a vehicle parking space. It should be appreciated that many rules areenvisioned.

In some embodiments, the user may access the calendaring functionalitydiscussed above using bot commands to reserve parking spaces in thefuture. For example, in an embodiment, the user of FIG. 5F may enter acommand such as, “check for open spaces tomorrow at 4 PM.” The usercommand may be processed by the bot module 124 of FIG. 1 . The botmodule 124 may analyze the user command to determine a user intent. Theuser intent may be determined to correspond to a scheduling command.Based on determining that the command is a scheduling command, thecommand may be transmitted to the scheduling module 182 of FIG. 1 . Thescheduling module 182 may execute the rules discussed, create an entryin an electronic calendar object, and transmit a notification to theuser indicating whether a spot is available.

Returning to FIG. 10 , the application 1002 depicts a dialogue betweenthe user seeking to reserve a parking space and the bot backend, in someembodiments. For example, the user input 1008-A indicates a user'sinitial space request. With respect to FIG. 1 and FIG. 10 , the botmodule 124 of FIG. 1 or the bot module 162 of FIG. 1 may analyze theuser input 1008-A and determine the user's intent corresponds to seekinga parking space. The bot module 124 may transmit the user input 1008-Ato the parking module 168, for example. The bot module 162 may transmitstatus messages (e.g., checking for spaces) to the client device 102while concurrently receiving output from the machine learning module178. For example, the bot module 162 may receive a queue count from thequeueing module 180 and transmit the user's place in the queue to theuser. For example, when the user is in the third position of a FIFOqueue, as depicted (e.g., number 3 in the queue), the user would be atposition 592-C of FIG. 5H. When the ML module 178 indicates no availablespaces, the bot module 162 may notify the user that no spaces areavailable, as depicted. For example, program output 1010-A includes thestatus message, the queue count, and the no-availability message.

In some embodiments, the bot module 124 may continue to monitor theoutput of the ML module 178 periodically, and when the ML module 178indicates the availability of an open space, the bot module 124 maytransmit a message such as program output 1010-B to the user, whereinthe user is solicited to provide an acceptance of the identified spot.The application 1002 may display a countdown, as depicted. When the userdoes not accept, or declines the spot, the bot module 124 may releasethe spot to another user by, for example, dequeueing another user fromthe queue 590. In the depicted example of FIG. 10 , the user inputs anaffirmative user input 1008-B. The bot module 124 may receive theaffirmative user input 1008-B, and transmit a response to the userconfirming that the spot has been reserved, as shown in program output1010-C.

ADDITIONAL CONSIDERATIONS

The following considerations also apply to the foregoing discussion.Throughout this specification, plural instances may implement operationsor structures described as a single instance. Although individualoperations of one or more methods are illustrated and described asseparate operations, one or more of the individual operations may beperformed concurrently, and nothing requires that the operations beperformed in the order illustrated. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

It should also be understood that, unless a term is expressly defined inthis patent using the sentence “As used herein, the term” “is herebydefined to mean . . . ” or a similar sentence, there is no intent tolimit the meaning of that term, either expressly or by implication,beyond its plain or ordinary meaning, and such term should not beinterpreted to be limited in scope based on any statement made in anysection of this patent (other than the language of the claims). To theextent that any term recited in the claims at the end of this patent isreferred to in this patent in a manner consistent with a single meaning,that is done for sake of clarity only so as to not confuse the reader,and it is not intended that such claim term be limited, by implicationor otherwise, to that single meaning. Finally, unless a claim element isdefined by reciting the word “means” and a function without the recitalof any structure, it is not intended that the scope of any claim elementbe interpreted based on the application of 35 U.S.C. § 112(f).

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of “a” or “an” is employed to describe elements andcomponents of the embodiments herein. This is done merely forconvenience and to give a general sense of the invention. Thisdescription should be read to include one or at least one and thesingular also includes the plural unless it is obvious that it is meantotherwise.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs forimplementing the concepts disclosed herein, through the principlesdisclosed herein. Thus, while particular embodiments and applicationshave been illustrated and described, it is to be understood that thedisclosed embodiments are not limited to the precise construction andcomponents disclosed herein. Various modifications, changes andvariations, which will be apparent to those skilled in the art, may bemade in the arrangement, operation and details of the method andapparatus disclosed herein without departing from the spirit and scopedefined in the appended claims.

What is claimed:
 1. A computer-implemented method for smart buildingmanagement using robot command and control, comprising: displaying, viaa graphical user interface, a building automation bot client applicationincluding an input field and an output field; receiving, via the inputfield, bot input from a user; analyzing the bot input from the userusing a trained machine learning model to identify a bot instruction;analyzing the bot instruction to identify a user intent; and causing,based on the user intent, a building automation task to occur.
 2. Thecomputer-implemented method of claim 1, wherein receiving, via the inputfield, bot input from the user includes receiving a spoken user command.3. The computer-implemented method of claim 1, further comprising:receiving, via the graphical user interface, a selection of the usercorresponding to an active channel; and updating an active channelindicator to correspond to the active channel selection.
 4. Thecomputer-implemented method of claim 1, wherein the user intent relatesto reserving a parking space, and further comprising: in response to thebot instruction, analyzing an image of a parking area using one or moretrained machine learning model to identify one or more available parkingspace.
 5. The computer-implemented method of claim 1, wherein the userintent corresponds to one or more of user authentication, ane-receptionist or kiosk-based user check in, determining a roompresence, a lighting control or a report.
 6. The computer-implementedmethod of claim 1, wherein the output includes a boolean questionrelating to the building automation task, and further comprising:receiving, in response to the boolean question, a user response via theinput field; and analyzing the user response and, when the response tothe boolean question is a positive response, causing the buildingautomation task to be performed.
 7. The computer-implemented method ofclaim 1, wherein the bot instruction corresponds to locating an electricvehicle charging location, and the one or more bot parameters include aport type and a mileage radius.
 8. A smart building management systemcomprising: one or more processors; and a memory storing instructionsthat, when executed by the one or more processors, cause the system to:display, via a graphical user interface, a building automation botclient application including an input field and an output field;receive, via the input field, bot input from a user; analyze the botinput from the user using a trained machine learning model to identify abot instruction; analyze the bot instruction to identify a user intent;and cause, based on the user intent, a building automation task tooccur.
 9. The smart building management system of claim 8, the memorystoring further instructions that when executed cause the system to:receive a spoken user command.
 10. The smart building management systemof claim 8, the memory storing further instructions that when executedcause the system to: receive, via the graphical user interface, aselection of the user corresponding to an active channel; and update anactive channel indicator to correspond to the active channel selection.11. The smart building management system of claim 8, the memory storingfurther instructions that when executed cause the system to: in responseto the bot instruction, analyze an image of a parking area using one ormore trained machine learning model to identify one or more availableparking space.
 12. The smart building management system of claim 8, thememory storing further instructions that when executed cause the systemto: receive a user intent corresponding to one or more of userauthentication, an e-receptionist or kiosk-based user check in,determining a room presence, a lighting control or a report.
 13. Thesmart building management system of claim 8, the memory storing furtherinstructions that when executed cause the system to: receive, inresponse to a boolean question, a user response via the input field; andanalyze the user response and, when the response to the boolean questionis a positive response, performing the building automation task.
 14. Thesmart building management system of claim 8, the memory storing furtherinstructions that when executed cause the system to: receive a botinstruction corresponding to locating an electric vehicle charginglocation, and one or more bot parameters including a port type and amileage radius.
 15. A non-transitory computer readable medium containingprogram instructions that when executed, cause a computer to: display,via a graphical user interface, a building automation bot clientapplication including an input field and an output field; receive, viathe input field, bot input from a user; analyze the bot input from theuser using a trained machine learning model to identify a botinstruction; analyze the bot instruction to identify a user intent; andcause, based on the user intent, a building automation task to occur.16. The non-transitory computer readable medium of claim 15, includingfurther program instructions that when executed, cause a computer to:receive a spoken user command.
 17. The non-transitory computer readablemedium of claim 15, including further program instructions that whenexecuted, cause a computer to: receive, via the graphical userinterface, a selection of the user corresponding to an active channel;and update an active channel indicator to correspond to the activechannel selection.
 18. The non-transitory computer readable medium ofclaim 15, including further program instructions that when executed,cause a computer to: in response to the bot instruction, analyze animage of a parking area using one or more trained machine learning modelto identify one or more available parking space.
 19. The non-transitorycomputer readable medium of claim 15, including further programinstructions that when executed, cause a computer to: receive a userintent corresponding to one or more of user authentication, ane-receptionist or kiosk-based user check in, determining a roompresence, a lighting control or a report.
 20. The non-transitorycomputer readable medium of claim 15, including further programinstructions that when executed, cause a computer to: receive, inresponse to a boolean question, a user response via the input field; andanalyze the user response and, when the response to the boolean questionis a positive response, causing the building automation task to beperformed.